Quick start on watsonx.ai LLM

watsonx.ai introduction

watsonx.ai is IBM's commercial generative AI and scientific data platform based on cloud technology. It encompasses a studio, data store, and governance toolkit, designed to support multiple LLMs. This platform is tailored for a wide range of AI development tasks, offering developers access to IBM's own series of LLMs as well as models from Meta's Llama-3 and Mixtral model.

Alt text

In this section, you will be guided through the process of using watsonx.ai's API to create a simple Q&A bot. This bot will leverage the advanced capabilities of watsonx.ai to understand and respond to user queries accurately. Whether you're new to programming or an experienced developer, this step-by-step tutorial will equip you with the knowledge to integrate watsonx.ai's LLMs into your applications.

Create a Q&A bot

This tutorial will walk you through the process of creating a Q&A chatbot leveraging the mistralai/mixtral-8x7b-instruct-v01 model developed by Mistral AI. This powerful foundation model has been seamlessly integrated into IBM's watsonx.ai platform, simplifying your development journey. The provided API eliminates the need for generating complex API tokens, streamlining the application creation process. The mistralai/mixtral-8x7b-instruct-v01 model is equipped with the following features:

  • Supports Q&A
  • Summarization
  • Classification
  • Generation
  • Extraction
  • Retrieval-augmented generation
  • Code generation

This model aligns perfectly with your objective for the application being created. For models and their IDs, please refer to here.

Note: The mistralai/mixtral-8x7b-instruct-v01 model, like any AI technology, has its limitations, and it is possible to encounter nonsensical responses occasionally. The primary objective of this project is to provide guidence on utilizing LLMs with watsonx.ai.

Follow these step-by-step instructions to create your application:

  1. Still in the PROJECT directory, create a new Python file named simple_llm.py (you are welcome to choose a different name if you prefer).

  2. Enter the following script content into your newly created simple_llm.py file and save your changes. Line-by-line explanation of the code snippet is provided.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  1. # Import necessary packages
  2. from ibm_watsonx_ai.foundation_models import ModelInference
  3. from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams
  4. from ibm_watsonx_ai import Credentials
  5. from langchain_ibm import WatsonxLLM
  6. # Model and project settings
  7. model_id = 'mistralai/mixtral-8x7b-instruct-v01' # Directly specifying the model
  8. # Set necessary parameters
  9. parameters = {
  10. GenParams.MAX_NEW_TOKENS: 256, # Specifying the max tokens you want to generate
  11. GenParams.TEMPERATURE: 0.5, # This randomness or creativity of the model's responses
  12. }
  13. project_id = "skills-network"
  14. # Wrap up the model into WatsonxLLM inference
  15. watsonx_llm = WatsonxLLM(
  16. model_id=model_id,
  17. url="https://us-south.ml.cloud.ibm.com",
  18. project_id=project_id,
  19. params=parameters,
  20. )
  21. # Get the query from the user input
  22. query = input("Please enter your query: ")
  23. # Print the generated response
  24. print(watsonx_llm.invoke(query))
  1. Open your terminal and ensure that you are operating within the virtual environment (my_env) you previously established.

  2. Run the following command in the terminal.

  1. 1
  1. python3.11 simple_llm.py

Upon successful execution of the code, you can input our query in the terminal. A response is then generated.

The following picture is showing an example response based on the query: How to be a good data scientist?.

Alt text

In the code, you simply used "skills-network" as project_id to gain immediate, free access to the API without the need for initial registration. It's important to note that this access method is exclusive to this cloud IDE environment. If you are interested in using the model/API in a local environment, detailed instructions and further information are available in this tutorial.